Ensembles of Abstaining Classifiers Based on Rule Sets

The role of abstaining from prediction by component classifiers in rule ensembles is discussed. We consider bagging and Ivotes approaches to construct such ensembles. In our proposal, component classifiers are based on unordered sets of rules with a classification strategy that solves ambiguous matching of the object’s description to the rules. We propose to induce rule sets by a sequential covering algorithm and to apply classification strategies using either rule support or discrimination measures. We adopt the classification strategies to abstaining by not using partial matching. Another contribution of this paper is an experimental evaluation of the effect of the abstaining on performance of ensembles. Results of comprehensive comparative experiments show that abstaining rule sets classifiers improve the accuracy, however this effect is more visible for bagging than for Ivotes.